1. Field of the Invention
The present invention relates to a technique in which a photographed real image and computer graphics are synthesized so that a specific image of the photographed real image is made as a foreground image.
2. Related Background Art
In the past, in the field of image production, such as a television program, a method has been used in which a specific area is cut out of a live image from a video camera using a chromakey method, and such area is synthesized with an image obtained by computer graphic, called a “CG image”, has been used.
Now, such a method will be explained with reference to FIG. 1.
In conventional chromakey synthesis, a photographic subject 103 is arranged in front of a screen 102 (hereinafter, referred as “blue back” regardless of color) having a specific color which is substantially not present in the photographic subject 103 (generally, a sharp blue or green color) and a photographed real image is acquired by a camera as a foreground image 101.
Then, from the foreground image 101, a key image 106 is generated using the color of the blue back 102 as a reference. The key image is information for cutting out the area of the photographic subject 103 included in the foreground image 101, and, as can be seen from FIG. 1, a value for distinguishing the background from the photographic subject is set for each pixel unit.
On the other hand, a background image 104 is an image such as a still image, a video moving image or computer graphic and, unlike the photography of the foreground image 101, the usage of the blue back is not required in the background image.
Image synthesis processing 107 is represented by the following equation. That is to say, in the key image, for example, a value of the area of the photographic subject is shown as 1 and a value of the area of the background is shown as 0. It is assumed that a symbol for representing a pixel value of the key image is K, a pixel value for representing the foreground image is F, a pixel value for representing the background image is B and a pixel value for representing an output image is C. In this case,If K=0, then, C=B, andIf K=1, then, C=F  (equation 1).
By such image synthesis processing 107, an image 108 is generated in which the photographic image 103 of the foreground image 101 is overlapped onto the background image 104.
As shown in FIG. 2, it is assumed that a virtual interior of a car, such as a CG steering wheel 204, is overlapped onto a photographed real scenery image 203 as the background, and a photographed real hand image 202 is synthesized as the foreground thereof.
In the above-mentioned conventional example, as shown in FIG. 1, only the foreground image 101 and background image 104 were considered and synthesis of three images as shown in FIG. 2 was not considered.
If the image shown in FIG. 2 is generated using the conventional technique, as shown in FIG. 3, the background image 203, CG steering wheel 204, CG inside car and photographed real hand image 202 must be prepared independently. In conventional chromakey processing, blue back 102 is required in order to photograph the photographic subject 103 as the foreground, and, thus, the photographic subject cannot be photographed simultaneously with the background image.
A problem in FIG. 3 is that, since the photographed real background image 203 and the photographed real hand image 202 are photographed by different cameras, if orientations and positions of the respective cameras are changed, a positional relationship deviates, and, thus, a natural-looking synthesized image cannot be obtained.